1. Field of the Invention
The invention relates to a practical means of using computer vision to control systems consisting of a combination of holonomic and nonholonomic degrees of freedom in order to perform user-designated operations on stationary objects. Examples of combination holonomic/nonholonomic systems are a wheeled rover equipped with a robotic arm, a forklift, and earth-moving equipment such as a backhoe or a front-loader and even an underwater vehicle with attached robotic arm.
The present invention eliminates the need for direct, ongoing human participation in the control loop for completing a given task such as engaging a pallet with a forklift. Whereas, depending upon the application of the present invention, the human may supply some high-level supervision for the system such as "engage pallet," the effect of the new art is to create fully autonomous response of the system, synchronized between control of the holonomic and nonholonomic degrees of freedom, which produces effective, precise, reliable and robust direction and control of the mechanism without any subsequent human intervention.
2. References
The remainder of this specification refers to various individual publications listed below by number by reciting, for example, "[1]", or "[2]", and so forth.
[1] E. Gonzalez-Galvan and S. B. Skaar, "Application of a precision enhancing measure in 3-D rigid-body positioning using camera-space manipulation," International Journal of Robotics Research, Vol. 16, No. 2, pp. 240-257, 1997.
[2] B. Horn, Robot Vision, MIT Press, Cambridge, 1986.
[3] M. Seelinger, S. B. Skaar, and M. Robinson, "An Alternative Approach for Image-Plane Control of Robots," Lecture Notes in Control and Information Sciences, Eds. D. Kriegman, G. Hager, and S. Morse, pp. 41-65, Springer, London, 1998.
[4] E. Gonzalez-Galvan and S. B. Skaar, "Servoable cameras for three dimensional positioning with camera-space manipulation," Proc. LASTED Robotics and Manufacturing, pp. 260-265, 1995.
[5] S. B. Skaar, I. Yalda-Mooshabad, and W. H, Brockman, "Nonholonomic camera-space manipulation," IEEE Trans. on Robotics and Automation, Vol. 13, No. 3, pp. 464-479, August 1992.
[6] R. K. Miller, D. G. Stewart, H. Brockman, and S. B. Skaar, "A camera space control system for an automated forklift," IEEE Trans. on Robotics and Automation, Vol. 10, No. 5, pp. 710-716, October 1994.
[7] Y. Hwang. "Motion Planning of a Robotic Arm on a Wheeled Vehicle on a Rugged Terrain," L. A. Demsetz, ed., Robotics for Challenging Environments, Proc. of RCE II, pp. 57-63, 1996.
[8] T. Lueth, U. Nassal, U. Rembold. "Reliability and Integrated Capabilities of Locomotion and Manipulation for Autonomous Robot Assembly," Robotics and Autonomous Systems. Vol. 14, No.2-3, pp. 185-198, May 1995.
[9] MacKenzie, D. and Arkin, R. "Behavior-Based Mobile Manipulations for Drum Sampling," Proceedings of the 1996 IEEE Int. Conf. On Robotics and Automation, pp 2389-2395, April 1996.
[10] C. Perrier, P. Dauchez, F. Pierrot. "A Global Approach for Motion Generation of Non-Holonomic Mobile Manipulators, " Proc. IEEE Int. Conference on Robotics and Automation pp. 2971-2976, 1998.
[11] O. Khatib, "Mobile manipulation: The robotic assistant," Robotics and Autonomous Systems, Vol. 26, pp. 175-183, 1999.
3. Nomenclature
The following is a summary of notation used in this specification:
C.sup.j =[C.sub.1.sup.j,C.sub.2.sup.j, . . . ,C.sub.6.sup.j ].sup.T view parameters for camera j
.THETA.=[.theta..sub.1,.theta..sub.2, . . . ,.theta..sub.n ].sup.T internal joint configuration of an n-degree of freedom system
(x.sub.c.sub..sub.i .sup.j, y.sub.c.sub..sub.i .sup.j) camera space location of point i in camera j
(f.sub.x,f.sub.y) orthographic camera model
J.sub.1,J.sub.2,J.sub.3 scalar quantities minimized to estimate various parameters
n.sub.cam number of cameras in system
n.sub.c (j) number of visual features used in any given summation
p number of poses in the pre-plan trajectory
W.sub.ik relative weight given to each visual sample